CHAPTER 3 



ASSESSMENT OF SKILL RETENTION 







3.1 Aim 

In this section of the thesis, studies have been made about the ability of 
school pupils to retain and reproduce practical skills involved in scientific 
work which have been taught earlier. 

The assessment of a pupil's ability to retain and reproduce knowledge and 
skill taught earlier could be taken as a measure of success in teaching. 
Information gained thereby regarding each pupil's peformance could be 
an important feedback material for rebuilding the future teaching strategy. 



To carry out this investigation, a group of pupils has been taught how to 
H carry out certain scientific manipulative skills and these were tested after 

,j| some time to find out how much they could retain and reproduce. 

Comparison of this result with the result of assessment of an untaught 
K %, group (control group) would help us to exclude the effect of general 
knowledge/ability gained by pupil by simply grdwing up in a modern 
society and spending time in the intellectual environment of a school, 
and allow us to find out the pupil's ability to retain knowledge acquired 
during teaching sessions, and to reproduce it, assuming that the teaching 
was carried out at the highest possible standards. 

3.2 Development of the study materials — Preliminary runs 

(i) Method 

In this part of the work in which the retention ability of pupils was 
investigated, a trial run was carried out only for the testing session and 
not for the teaching session. A group of 5 pupils (14 years old) was invited 
to take part in this trial run. We were able to find out how many solid 
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samples each pupil could study in half an hour and how many runs of the 
second experiment (pH of the mixture of weak acid and weak alkali) were 
possible in half an hour •• •" ■ ■■■■-* 

In general the objective was to find out if the preparation of the work- 
|| sheet and arrangement of the materials was adequate for. the testing 
% session. 

Er (H) Discussion of the findings of the trial run 

No major (or significant) changes were required. The trial run was carried 
' but as planned and the same procedure was repeated for the testing 

?■;■;'' sessionsl . ' . ■ v :■•.;■.•■ I ■ 

3.3 Final run 

(i) Method > 

1 . . In this section of the study, work was carried out in two stages; the first : 

stage involves the teaching of practical skills and the second stage 
|| involves testing the pupils' ability to reproduce these skills after a gap of a 

if certain period of time. 

Four sets of fourth form pupils from a locai comprehensive school 
(Earlham School, Norwich) took part in this study. There were 
approximately 25 pupils (boys and girls of age range 14-15 years) in each 
% set. 

For each of the four teaching sessions, organised at the end of their 
summer term, the pupils in each set were randomly divided into 3 
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groups. Group I was asked to carry out an experiment on the reaction of 
dilute hydrochloric acid (HC1) with sodium thiosulphate solution 
(Na 2 S20g) [see work-sheet 3.1 in Appendix 2.1].' Pupils in Group II were 
given a series of solid samples in ignition tubes and asked to heat them on 
a Bunsen burner and record their observations [see work-sheet 3.2 in 
Appendix 2.1]. All the materials needed for the experiments were 
supplied at the desk before the start of the work. Group III pupils watched 
a video film in which none of the skills used in the above mentioned 
experiments were involved. 

For this experimental work, one large laboratory in Earlham School was 
used. The video film was shown in a small room adjacent to the 
laboratory. The laboratory floor was divided into 2 sections, one section 
contained the acid - thiosulphate solution experiment involving volume 
measurement by measuring cylinder, while the other section contained 
the heating of solids experiments. " ^ 

All the necessary materials were prepared and arranged at the UEA earlier. 
On the day before the teaching session, these materials were moved to the 
school and laid out on the bench. In between the sessions, the chemical 
bottles were refilled and new sample tubes for heating experiments were 
placed on the desk. Lists of chemicals and apparatus used in experiment 
3.1 and 3.2 are given in Appendix 2.2. 

Pupils in Group I and II, engaged in the experimental work, were 
supervised by four science teachers (two for each group) who spent about 
one hour teaching various manipulative skills involved in the 
experiment. 
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The Group I pupils were taught how to measure accurately a certain 
volume of liquid using a measuring cylinder. They were asked to ensure 
that the lower meniscus of the liquid touched the required mark on the 
cylinder. For heating solids by Group II pupils, the following four 
manipulative skills, viz. (a) the position of the test tube holder, (b) angle of 
the test tube, (c) zone of the flame used in heating, and (d) colour of the 
flame, were thought to be important and the pupils were given thorough 
lessons on them, so that they could perform these skills correctly. The 
pupils were told that (i) the test tube holder should be kept close to the top 

end of the tube, (ii) the test tube should be held at an angle of about 45° 

and not pointing towards any person, (iii) the sample should be in the hot 
zone of the flame during heating, and (v) a blue coloured flame should be 
used for heating. 

> 
After two months of summer holidays, at the beginning of a new term, 

each of these pupils was asked to take part in experimental work for an 

hour and was given two experiments different from the previous ones, 

but involving the above-mentioned skills. The two experiments were: (i) 

the mixing of weak acid and weak alkali in different volume ratios and to 

determine the pH value of the mixture by using an indicator solution (see 

work sheet 3.3 in Appendix 2.1), and (ii) the heating of solids in ignition 

tubes and identification of the nature (acid/neutral/ alkaline) of the gas 

given off (see work sheet 3.4 in Appendix 2.1). The first experiment 

provided the opportunity for assessing the volume measurement skill 

while the second experiment provided the opportunity for assessing the 4 

manipulative skills involved in heating solids in a tube, as has been 

mentioned earlier in the teaching session. 
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;;..... It was the testing-time, i.e. the time to find out how much a pupil could 

remember and reproduce the skills which were taught earlier, and also to 
£' look at the performance of the pupils who did not receive lessons on any 
skill, in comparison with that of the pupils who were taught. For this 
purpose each set of about 25 pupils comprising samples from each of the 3 
|rj; - groups (i.e. measurement of volume group, heating group and video 
, group/ control group) was assessed in each session lasting for about an 
- hour. 

For this work, a spacious laboratory in Earlham School was used. It was 
the same laboratory as was used for the teaching sessions about two 
months previously. 

The numbering of the work-stations, randomisation of the pupils, and 
their identification with number labels was carried out in the same way as 
has been described earlier for the previous experiment (experiment 2.1) in 
Chapter 2. 



Materials required for both the experiments were laid out on the desk the 
previous day and some of the used chemicals were replaced in between 
the sessions. Arandix 2.2 contains the lists of chemicals and apparatus 
used in experiment 3.3. and 3.4 There were three sessions to assess three 
sets of pupils. Although four sets of pupils were given lessons, it was 
possible to assess only the three sets as the judges had to leave the 
laboratory to attend to other business before the commencement of the 
[ ■ fourth session. However, for data analysis, it did not cause any difficulties 

as each set of pupils had almost an equal number of samples from the 
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three method groups (volume measurement, heating solid and control). 

Each session, lasting for about an hour, was divided into two halves; one 
for the experiment involving measurement of volume and the other for 
the heating of solids in test tubes. Eight experienced science teachers acted 
|; "1 as judges. In the first half of the session, each judge assessed the pupils' 
skill to measure volume of liquid using a measuring cylinder and in the 
second half, the assessment of the four skills involved in heating solids. 

At the beginning of each assessment session, each of the 8 judges was 
provided with an assessment sheet (see appendix 2.1) to record the marks 
for the assessment of the five manipulative skills, one skill in the first 
; experiment (experiment 3.3) and four skills in the second experiment 
(experiment 3.4) for each pupil. "1" mark was awarded for correct 
execution of each skill and a: "0' mark for failing to do so. As a result 
each pupil obtained a maximum of 8 marks in experiment 3.3. (see work- 
sheet 3.3 in Appendix 2.1) and 32 marks in experiment 3.4 (see work-sheet 
3.4 in Appendix 2.1) for correct execution of all the manipulative skills. 

On the day before the assessment sessions took place, the judges were 
briefed clearly about the nature of their tasks and, following a discussion, 
they agreed upon a standard similar to that maintained during the 
teaching sessions, regarding the correctness of the manipulative skills. As 
the judges were different from the teachers involved in the earlier 
teaching sessions, they had no knowledge of the grouping of the pupils 
and this helped them to assess the pupils in an unbiased manner (without 
knowing who had lessons on what skill). 
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3.3. (ii) Results and Discussion 

The achievement of pupils belonging to all the three groups in each of the 
three sessions in carrying out the practical skills involved in two 
experiments have been determined by collating the results of the 
assessment from 8 judges' mark-sheets. 

Appendix 2.3 contains the "raw-data", i.e. the judges' mark sheets for all 
three sessions of the assessment. 'V"and " %" in the mark-sheets 
indicate the correct and incorrect execution of the manipulative skills. 
Assigning mark "1" for "/" and mark "0" for "X", the total marks 
(achievement) obtained by each pupil from 8 judges were calculated and 
are given in Table 3.1 (for session -I), Table 3.2 (for session -2) and Table 3.3 
(for session -3). As each session comprised an almost equal number of 
pupils belonging to the 3 method groups (volume group, heating group 
and control group) and following the randomised division during the 
teaching period earlier, the results of all 3 sessions were used in the 
analysis to calculate an average achievement value of the pupils 
belonging to each method group and for carrying out 5 skills (one in 
experiment 3.3 and four in experiment 3.4). 



The mean values of achievment obtained from the analysis of the 
experimental data are presented in Table 3.4. 



In experiment 3.3, which involves the measurement of volume, pupils in 
group I who had lessons in this skill have scored significantly better than 
the other two groups, whose achievements are very similar to each other. 
In experiment 3.4, pupils who had lessons on heating solids, have 
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produced, on the whole, (i.e. considering all the skills involved) better 
results than the other two groups. As separate skills are involved in 
experiment 3.4, the results were further analysed to understand "more 
clearly the pupils' performance in each of the skills. For the two tasks 
concerned with the position of the test-tube holder and angle of the tube, 
there is no significant differer^ (considering the s.d) between the 
achievement of each group in carrying out these two skills. For the task 
concerned with the zone of the flame, each group, even the taught one, 
produced poor results suggesting that, teaching of the skill had no 
permanent effect on the pupil. 

However, in the task concerned with using the correct colour 6f; flame, all 
three. groups have very high mean scores. Thus it would have been 
extremely difficult to improve on their answers by this teaching. 

The results indicate that the pupils who were given lessons on a practical 
skill can perform this skill better than those who have not, even after two 
months holiday, during which they did not take part in any academic 
activity involving the practical skill. In experiment 3.3 the taught group is 
twice as good as the untaught group, while in experiment 3.4 the taught " 

group is slightly better than the untaught groups. 

i 

In order to understand more clearly the effect of the variables on the 
variability in score discussed above, further analysis of the pupils' 
achievement data was carried out. 

Several 3X3 AN OVA based on randomised groups were done. 
They were as follows: 
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i) In experiment (3.3):- 3 method group (A) X 
3 sets (B) 

ii) In experiment (3.4), 

skill (a):- 3 method group (A) X 3 sets (B) 
iii) In experiment (3.4), 

skill (b):- 3 method group (A) X 3 sets (B) 

(iv) In experiment (3.4) 

skill (c):- 3 method group (A) X 3 sets (B) 

(v) In experiment (3.4) 

skill (d):- 3 method group (A) X 3 sets (B) 

Information derived from the ANOVA is in Tables 3.5. 

It seems from the analysis of variable (F^ is significant at p/b.0001 level) 

that in experiment 3.3, the teaching of volume measurement has a 
significant effect on the final results discussed earlier. The significance of 
Fa values also indicates that in experiment 3.4, the teaching of two skills, 

viz. (a) position of the test tube holder, and (b) angle of the tube-tube, has a 

significant effect on the final results; whereas teaching of other two skills, 

viz. (c) zone of the flame, and (d) colour of the flame, has no significant 
effect on the final results. The statistical analysis of pupils' achievement 
data support the conclusion drawn earlier regarding the variability in 
score due to different conditions in the variable (i.e. method group: taught 
group vs. untaught group). On the basis of Fg or F^XB value, it can be 
suggested that the overall results gathered were not affected significantly 
due to the fact that the subjects (pupils) belonged to three sets and they 
were assessed in three separate sessions. 
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Table 3.1 . 

Total marks obtained by. each pupil' assessed.. by 8 judge's in Session 1 
for carrying out volume measurement in. experiment 3'. 3 and heating 
solids in experiment "3. 4. 






Pupil No. 


Experiment 

3.3 

max marks 
= 8 




Experiment' 3>4 






Position of 

test tube.holder 
max .marks 

= 8 


Angle of 
test tube 
max marks 
• = 8 


Zone of 
flame 
max marks 
= 8 


Colour of 
flame 

max marks 
= 8 


1 H 


3 ' 


6 


8 


3 


7 


7 


- 


- 


■ v.. — 


V ; ... 


-'-. 


3 


- 


- 


_ 


_ 


,L_ 


4 V 


7 


7 


6 


1 


5 


5 V 


8 


7-' 


8 


5 


8 


'■&■:■ ... . - 


- 


,;.• ■<.;-.- 


,,:■■ r.: T- ■.-; 


.' , '• ' '-. '■■'.'- '■ 


;';;':■ ■''.•*" 


7 c ■- 


2 


8 


6 


.3 , 


.8 


8 c • 


2 


5 


7 


1 


8 


9 c .. 


4 





8 


8" 


8 


10 v « 


8 


2 


4 


6 


8 


.11 H 


2 


7' 


■ '■' 8 ■> 





8 


12 v 


5 


3 


■ 5 


. , ■ 1 ■-', :■ . 


.-•■' '8 • - 


13 H 


4 


7 


■8 • 


8 


8 


14 v 


6 


5 


" 7 


5 


8 


15 H 


1 


7 


5 


5 


8 


16 H 


7 


7 


8 


3 


8 


17. c 


7 


8 


7/ 


6 


6 


18 


- 


- 


- 


- 


_ 


19 


- 


' 


- 


- 


- 


20 


- 


- 


- 


- 


- 


21 v 


8 


3 


3 


7 


7 


22 v 


3 


7 


4 


7 


6 


23 c 


7 


4 


8 


5 


8 


24 H 


. 


8 


7 


7 


8 


25 c 


5 


7 


7 


6 


7 



V/h/C in Column one indicate the group (volume measurement/heating 
solids/control) they belonged to during the teaching sessions earlier. 



Table 3.2 
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Total marks obtained by each pupil assessed by 8 judges in Session 2 
for carrying out volume measurement in experiment 3.3 and heating 
solids in experiment 3.4. 



i 
SI 



Pupil No. 


Experiment 
*3.3 

max marks 
= 8 




Experiment 3 . 4 




Position of 
test tube, holder 
max marks 
= 8 


Angle of 
test tube 

max marks 
= 8 


Zone of 
flame 

max marks 
= R 


Colour of 
flame 

max marks 

= fl 


1 H _ 


5 


7 


7 


4 





?. 


- 


- 


- 


- 


- 


3 c 


2 


2 


4 


3 


7 


4 c 


6 


3 


7 


1 


8 


5 H 


7 


7 


7 





8 


6 H 


• , 5 . 


8 


8 


3 


■ 8 • 


7 H 


1 


7 


7 





7 


8 c 


4 


5 


6 


2 


8 


9 


- 


— 


_ 


— 


— 


10 v 


7 


4 


7 


5 


8 


n'.y 


8 


7 


■> 
■ 1 


1 


8 


12 c 


2 


2 


3 


2 


8 


13. H 


3 


5 


7 


6 


8 


14 


- 


- 


_ 


*. 


— 


15 c 


1 


6 


8 


7 


8 


16 H 


4 


7 


8 


6 


8. 


17 v 


7 


7 


8 ; 


6 


8 


18 y 


8 


7 


8 


3 


8 . 


19 v 


7 


"7 


3 


1 





20 H 


1 


3 


6 








21 c 


5 


3 


2 


1 





22 c 


4 


2 


5 


5 


3 


23 v 


8 


8 


3 


5 


5 


24 v 


7 


8 


6 


6 


• 5 


25 


- 


- 


- 


- 


- 



V/h/c in Column one indicate the group (volume measurement/heating 
solids /control) they belonged to during the teaching sessions earlier. 
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Table 3.3 

Total marks obtained by each pupil assessed by 8 judges in Session 3 
for carrying out volume measurement iin experiment 3.3 and heating 
solids in experiment 3'. 4. 



i'pupil.No. 


Experiment 

max marks 
= 8 




Experiment 3 . 4 






Position of 
test tube holder 
max marks 
= 8 


Angle of 
test tube 

max marks 
= 8 


Zone of 
flame 

max marks 
= 8 


Colour of 

flame 
max marks 

= 8 


~1 H 





6 


5 


1 


7 


n, V 


7 


2 


2 





7 


3 v 


1 


3 


4 


1 


8 


% v 


6 


4 


2 


1 


6 


fi. c 


2 


2 





,5 


2 


6 c 


- 3 


6 .; 


,7. 


2 


8 ....... 


7 c 


3 


3 


•3 


5 


8 


.8 c 


i 


' -7 ■ 


6 


3 


8 L 


9 


- 


- 


- 


- 


- 


10 v 


5 


1 


4' 


1 


8 


11 c 


3 


7 


8 


> 2 


4 


12 c ■ 


3 


7 


.5 





8 


13 v 


5 


3 


4 


2 


8 


14- c 


2 


8 


1 


2 


8 


Is v 


3 


4 


1 


2 


8 


1'6 v 


4 


3 


2 





d 


17 H 


1 


7 


7 , 


2 


8 


:18 


- 


- 


- 


- 


- 


19 H 


5 


7 


6 


5 


4 


20 H 


5 


7 


8 


7 


7 


21 


- 


- 


- 


- 


- 


22 c 


'1 


7 


6 


5 


8 


23 H 


5 


8 


7 


8 


8 


24 


- 


- 


- 


- 


- 


25 H 


5 


7 


7 


8 


8 



V/h/C in Column one indicate the group (volume measurement/heating 
solids/control) they belonged to during the teaching sessions earlier. 



72 



Table 3.4 

Mean values of the achievement of different groups of pupils in carrying out several 
practical skills in two experiments 



Expt. Practical skill 
assessed 



Measurement of 
volume group 

x s.d. 



Heating of solids 
group 

x s.d. 



Control 
group 

x s.d 



33 Measurement of 

volume 



6.10 +1.98 



3.37 ±2.18 3.28 ±1.80 

> 



3.4 (a) Position of the 

test tube holder 



4.86 +2.21 



6.74 +1.12 4.86 +2.42 



(b) Angle of the 
test tube 



4.24 +2.22 



7.05 +0.94 5.43 +2.34 



(c) Zone of flame 



3.14 +2.42 



4.00 +2.87 3.52 +2.17 



(d) Colour of flame 6.90 ±1.90 



6.74 +2.49 6.71 +2.31 



(x = mean value, s.d. = standard deviation) 
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Table 3.5 
tables for 3X3 completely randomised ANOVA of the results of 

pils' ability to retain scientific manipulative skills 



" eriment 3.3 and 3.4. jAll the assessments were carried out by 



Rfethod -T) 



f/(a)' Experiment 




^ (3 method group) 
' : f(3 class/set) 



jAError 

ttfotal 



3.3 Measurement of volume using measuring cylinder 



SS 



df 



MS F F 
(obsv.) (crit) 



Significance of 
F (obsv.) 



98.26 


2 


23.60 


2 


24.85 


4 


160.50 


45 


307.21 


53 



49.13 

11.80 

6.21 

3.56 



13.80 8.25 at Significant at 

(p=0.001) p <^ 0.001 

3.31 3.23 at Significant at 

(p=0.05) p ^ 0.05 

1.74 2.61 at n.s. (p ^ 0.05) 
(p=0.05)- ' 



IB 



IbK'Experiment 3.4 Heating of solids (skill tested: Position of the 
iPrft,: - test tube holder) 



& Source of 



iavariance 



|At( 3 method group) 



E_B (3 sets) 



_ 

fcrror 
--Total 



SS 


df 


MS 


F > 
(obsv.) 


F 
(crit) 


Significance of 
F (obsv.) 


60.77 


2 


30.39 


10.16 


8.25 at 
(p=0.001) 


Significant at 
p <J 0.001 


4.11 


2 


2.06 


0.69 , 


3.23 at 
(p=0.05) 


n.s. (p ^ 0.05) 


53.78 


4 


13.45 


4.50 


3.83 at 
(p=0.01) 


Significant at 
p <[ 0.01 


134.67 


45 


2.99 


l- 


- 




253.33 


53 


- 


- 


- 


- 



pXc). Experiment 3.4 Heating of solids (skill tested: Angle of the test tube) 



IIP" 

teSource of 
gyariance 



SS 



df 



MS F F 
(obsv.) (crit) 



Significance of 
F (obsv.) 



;. A (3 method group) 
: B (3 sets) 
'■A'X B 

E rror 
T ot a i 



72.26 
26.93 
13.96 



2 
2 

4 



150.54 45 
277.65 53 



36.13 

13.46 

3.49 

3.35 



10.79 8.25 at Significant at 

(p=0.001) (p < 0.001) 

4.02 3.23 at Significant at 

(p=0.05) (p < 0.05) 

1.04 2.61 at n.s. at 

(p=0.05) (p < 0.05) 
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Table 3.5 
'"" Experiment 3.4 Heating of solids (skill tested: Zone of flame) 



f$pu 



|rce 



mi 



fiance 



SS~ - df MS F F Significance of 

(obsv.) (crit) F . (obsv.) 

at p=0.05 



^3 method group) 
(3 sets) 



14.93 

22.26 

37.07 

261.17 

335.43 



2 

2 

4 

45 

53 



7.46 

11.13 

9.27 

5.80 



1.29 3.23 

1.92 3.23 

1.60 2.61 



n.s. (p <^ 0.05) 
n.s. (p / 0.05) 
n.s. (p / 0.05) 






jfje) Expe 



riment 3.4 Heating of solids (skill tested: Colour of flame) 



Source of 

tfariance 



SS df MS F F 

(obsv.) (crit) 

at p=0.05 



fa (3 method group) 
IB (3 sets) 
aXb 

||rror 
trotal 



1.02 

12.26 

4.32 

218.33 

235.93 



2 
2 

4 
45 
53 



0.51 
6.13 
1.08 
4.85 



0.11 3.23 
1.26 3.23 
0.22 2.61 



Significance of 
F (obsv.) 



n.s. (p <^ 0.05) 
n.s. (p <( 0.05) 
n.s. (p ( 0.05) 



;lii] tables 3.5 (a) 
jffe(obsv:) 

Blrit) 



- (e) 



sfcs. 



denotes the value of F ratio obtained from the 
experimental results . 

denotes the critical value of F ratio at a given level 
of significance (i.e. p value) arid at the appropriate 
df values (see reference 50 for the relevant statistical 
tables) . 

means "not significant". This refers to the fact that the 
value of observed F ratio is less than the critical value of 
F at p=0.05 and thus F (observed) is not significant at 
p<^ 0.05 level. 

However, when F (observed) is higher than the F (critical) 
value at p=0.05, 0.01 or 0.001, the F (observed) is thought 
to be significant at the corresponding level of significance 
(p-value) . 



